I know in different areas of the country Jehovah's Witnesses look at gun ownership differently. If you have guns strictly to acquire food they don't say much but if you own hand guns in our area they will not appoint them as servants. Many JW's would never admit they own guns because they are pacifists and it is actually frowned upon by many.
You may want to write a letter to the local congregation and let them know what is going on and just let them know there are many neighbors upset with it and you were hoping they could assist in the matter. If they get a complaint they will talk to the head of that household. I would never sign my name to it though because you never know who they are connect to and if they will protect them.
You may want to contact an attorney first to see what your rights are and if they are violating the law. I hope you can get this worked out so you can have some peace.